Model Evaluation: Classification Report¶
To get a more detailed overview of how the model performed, we can build a classification report that will compute the F1_Score, the Precision, the Recall, and the Support. Now, I'm assuming you don't know what these metrics are, so let's take some time to go over them.
Accuracy:¶
Accuracy measures the portion of all testing samples classified correctly and is defined as the following:
\begin{align}
\\
\ Accuracy = \frac{tp + tn}{(tp +tn)+(fp-fn)}
\\
\end{align}$$
\begin{align}
\ where,
\\
\ tp = \text{True Positive}
\\
\ tn = \text{True Negative}
\\
\ fp = \text{False Positive}
\\
\ fn = \text{False Negative}
\\
\end{align}
$$
Recall¶
Recall (also known as sensitivity) measures the ability of a classifier to correctly identify positive labels and is defined as the following:
$$
\begin{align}
\\
\ Recall = \frac{tp}{(tp +fn)}
\\
\end{align}
$$$$
\begin{align}
\ where,
\\
\ tp = \text{True Positive}
\\
\ fn = \text{False Negative}
\\
\end{align}
$$
The recall is intuitively the ability of the classifier to find all the positive samples. The best value is 1, and the worst value is 0.
Specificity¶
Specificity measures the classifier’s ability to correctly identify negative labels and is defined as the following:
$$
\begin{align}
\\
\ Specificity = \frac{tn}{(tn +fp)}
\\
\end{align}
$$$$
\begin{align}
\ where,
\\
\ tn = \text{True Negative}
\\
\ fp = \text{False Positive}
\\
\end{align}
$$
Percision¶
Precision measures the proportion of all correctly identified samples in a population of samples which are classified as positive labels and is defined as the following:
$$
\begin{align}
\\
\ Percision = \frac{tp}{(tp +fp)}
\\
\end{align}
$$$$
\begin{align}
\ where,
\\
\ tp = \text{True Positive}
\\
\ fp = \text{False Positive}
\\
\end{align}
$$
The precision is intuitively the ability of the classifier not to label as positive a sample that is negative. The best value is 1, and the worst value is 0.
Interpreting the Classification Report¶
Now the fun part, interpretation. When it comes to evaluating the model, we generally look at the accuracy. If our accuracy is high, it means our model is correctly classifying items.
In some cases, we will have models that may have low precision or high recall. It's difficult to compare two models with low precision and high recall or vice versa. To make results comparable, we use a metric called the F-Score. The F-score helps to measure Recall and Precision at the same time. It uses Harmonic Mean in place of Arithmetic Mean by punishing the extreme values more.
